1. Field of the Invention
The present invention relates to an image capturing apparatus and, more particularly, to an image capturing apparatus which automatically captures an image in accordance with image capturing conditions. The present invention also relates to an image capturing technique of simultaneously photographing moving and still images.
2. Description of the Related Art
There has conventionally been known a technique associated with an image capturing apparatus which automatically captures an image in accordance with image capturing conditions.
Japanese Patent Laid-Open No. 2000-259833 discloses a technique of determining the state of a person's face from successive images and selecting an image exhibiting an optimal face state.
Japanese Patent No. 3240339 discloses a technique of, when an image at a timing desired by the user does not exist, performing interpolation processing using pixel correspondence between different images for recorded moving image data, and thereby generating an image at the timing desired by the user.
There has also conventionally been known a technique of tracking the motion of an object and photographing the object.
Japanese Patent No. 3200950 discloses a technique of extracting a partial template from an image and tracking an object on the basis of the correlation between the template and an input image. In tracking, the template is divided, integrated, and updated on the basis of the tracking log and the knowledge of the object. The behavior pattern of an object is learned, and the correlation calculation range is determined on the basis of future behavior prediction.
There is known a technique of, when detecting the position of a target object on the basis of the minimum residual between an input image and reference image data, detecting an operation (speed) based on two positions of the target object that are detected in time series or an acceleration based on three positions, and further calculating a predicted position. Still another technique is known for a correlation tracking system having a means for finding out a template of the largest coincidence from a plurality of templates for a target whose shape changes, sequentially calculating a motion vector to the position of the template, and tracking the target.
According to the above-described techniques, an image in an optimal face state can be determined and selected from a string of images stored in a memory after the images are input.
However, the timing when an image in an optimal state is input cannot be set or predicted. Thus, in order to select an image, unnecessary image data must be recorded, requiring a large-capacity memory. It is difficult to satisfactorily track an object state which changes at high speed, and obtain a high-quality image at a normal recording rate. Further, interpolation does not always provide an image at a desired timing.
It is difficult to track a moving object when the illumination condition changes or the image capturing apparatus itself moves.
In a configuration disclosed in Japanese Patent No. 3200950, an object is tracked on the basis of the correlation between the template and a portion extracted from an input image. However, it is generally hard to properly set a partial area to be extracted from an input image. When a tracked object is out of focus upon variations in photographing conditions or the illumination condition varies, the performance to track an object which moves and changes at high speed cannot be maintained. If a pattern of a motion vector similar to an object during tracking exists on the background, the background object may be erroneously recognized and tracked.
On the other hand, there has conventionally been known a hybrid camera capable of photographing by switching between moving images and still images higher in pixel count and image quality than moving images.
As a technique capable of recording a high-pixel-count still image during recording of a moving image or the like at a low pixel count, high-pixel-count still image is recorded by pressing the mode button, as described in, e.g., Japanese Patent Laid-Open No. 2003-125344 (FIG. 22).
There is also known a technique of switching the photographing mode to a high-quality mode externally by wireless communication as described in Japanese Patent Laid-Open No. 11-069222 (FIG. 23). If the shutter button is pressed in step S61010 of FIG. 22, recording is done with a normal moving image frame in FIG. 24 (320×240 pixels in FIG. 24). If the high-quality mode button is pressed in step S61014, a frame of high-pixel-count recording (1,280×960 pixels in FIG. 24) is inserted between moving image frames. In FIG. 23, an external controller 6107 switches between high-speed photographing and high-quality photographing by controlling a recording mode control unit 6104 and photographing control unit 6105 of an image capturing apparatus 6101 via a wireless communication unit made up of a reception unit 6103, antennas 6106 and 6111, and a transmission unit 6110.
As a method of determining a photo opportunity for more desirable photographing, there is proposed a method of automatically determining a photo opportunity by a camera to take a picture when an object shows a desirable facial expression or pose, as described in Japanese Patent Laid-Open No. 2004-294498 (FIG. 25). In the example of FIG. 25, an image capturing unit 6202 repetitively acquires images. When a control unit 6209 determines that a preset condition is satisfied, it controls the focus and diaphragm of the image capturing unit 6202 to take a picture. Note that the photo opportunity is determined by extracting a shape representing the facial expression of a person or the like from an image, calculating the coincidence between the extracted shape and a predetermined shape, and when the coincidence exceeds a predetermined value, determining the current timing as a photo opportunity to photograph an object.
There is a demand to record, at high image quality without influencing recording of a moving image, a photo opportunity for an instantaneous facial expression which appears during moving image photographing. For this purpose, according to the techniques disclosed in Japanese Patent Laid-Open Nos. 2003-125344 and 11-069222, moving image photographing can switch to high-pixel-count still image photographing. The user waits for a photo opportunity while photographing a moving image, and can photograph a desirable still image.
However, a time lag is generated in switching from the moving image photographing mode to the still image photographing mode, and the user may miss a photo opportunity in pressing the shutter button. As a result, photographing likely fails because an advanced technique of, e.g., predicting a change and releasing the shutter is required to prevent an image in which the eyes of an object are shut due to an instantaneous blink, or to capture the moment of a good facial expression such as a smile. In Japanese Patent Laid-Open No. 11-069222, moving image photographing is interrupted by high-quality photographing, so moving image recording is intermitted for a long time.
In Japanese Patent Laid-Open No. 2004-294498, the camera automatically determines a photo opportunity, and the user himself need not release the trigger at a photo opportunity. The camera determines the next photo opportunity from images photographed in the past and shifts to a photographing operation. Thus, a time lag in the operation of the AF, diaphragm, or the like, or a time lag such as the time difference between frames is generated, and an instantaneous photo opportunity may be missed. In addition, no moving image can be photographed at the same time as a still image because the image capturing unit 6202 is occupied for photographing of the still image.
A two-lens configuration is also conceivable in which a moving image photographing system and a high-quality photographing system including a system for detecting a face such as a facial expression are designed as separate optics and stored in one housing. However, the apparatus becomes bulky, and it is difficult to control the two optics in synchronism with each other. Moreover, the two optics do not always coincide with each other in view angle, focus status, and F-number, and an intended image is less likely obtained.